Communication-Efficient and Byzantine-Robust Distributed Learning With Error Feedback

We develop a communication-efficient distributed learning algorithm that is robust against Byzantine worker machines. propose and analyze gradient-descent performs simple thresholding based on gradient norms to mitigate failures. show the (statistical) error-rate of our matches Yin et al. (2018), which uses more complicated schemes (coordinate-wise median, trimmed mean). Furthermore, for communication efficiency, we consider generic class $\delta $ -approximate compressors from Karimireddi (2019) encompasses sign-based top- notation="LaTeX">$k$ sparsification. Our compressed gradients aggregation removal respectively. establish statistical error rate non-convex smooth loss functions. that, in certain range compression factor , (order-wise) convergence not affected by operation. Moreover, descent with feedback (proposed 2019) setting presence exploiting improves rate. Finally, experimentally validate results good performance convex (least-square regression) (neural network training) problems.